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g "That Was Me!": Applications of the 

N Soundbeam" MIDI controller as a key to 
g creative communication, learning, independence 

and joy. 

Tim Swingler B.Sc. (hons). Cert Ed, DSA. 

TTie Soundbeam Project 
United Kingdom 

Paper presented at the CSUN conference Technology and Persons with Disabilities Los Angeles, 
March 1998 

ABSTRACT 



Recent advances in movement sensor and other technology have utilised various electronic media to 
create human - machine interfaces with apparently significant but largely unexplored educational 
potential. Such systems have often been conceived and developed for purposes outside the sphere of 
formal school education or therapy (e.g. performance art, dance, electroacoustic composition, VR 
environments, video animation), yet preliminary research demonstrates that it is the school environment 
in which such technology can contribute most dramatically; specifically to the educational experience of 
young children with severe disabilities who may have missed the foundation stones of early learning and 
for whom a first experience of enjoyable control and initiation can be a crucial educational motivator. 



‘Aesthetic resonation’ is a term coined by Dr Phil Ellis to describe special moments experienced by 
individuals described as having profound and multiple learning difficulties, in which they achieve total 
control and expression in sound after a period of intense exploration, discovery and creation. Enjoyment 
and self-motivation are key aspects of this work, which Ellis describes as Sound Therapy. 
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Soundbeam" is one example of such technology. It emits an ultrasonic beam of variable range. 
Movements within the beam generate data interpreted by any MIDI instrument. Essentially, its an 
invisible, elastic keyboard in space that allows sound to be created without the need for physical contact 
with any equipment. Originally developed for dancers - giving them a redefined relationship with music 
- Soundbeam has proved to have dramatic significance in the disability field, because even with 
profound levels of impairment the most minimal movements can instigate and shape interesting sound 
effects, trigger rich and exotic aural textures, or perform soaring improvisations. Individuals especially 
difficult to stimulate can benefit from what may for them be a first experience of initiation and control. 
The core objectives of this presentation are: 

• to introduce a new technology which is used in over 1 000 special education and adult centres in 
over twenty countries; 

• to explain the evolution of this technology, focusing on the development of The Soundbeam 
Project as an unusual partnership between professionals in special education, music composition, 
electronics and psychology; 

• to investigate the operation and performance of the equipment; 

• to examine, using video and published academic research literature, a continuum of applications, 
ranging from the basics of cause-and-effect work and 'sound therapy', through to more ambitious 
creative/performance applications; 
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to allow delegates to make an informed evaluation of the system. 



INTRODUCTION 

The focus of this presentation is a specific item of technology - Soundbeam - which (at least in theory) 
makes the universal medium of music newly accessible, particularly to those who, because of disability, 
might hitherto have been denied independent creative experience. 

The idea of a musical instrument which could be played without any physical contact was first 
developed by Leon Theremin, the Russian composer whose 'Thereminvox' astounded audiences in the 
1920's, and which can be heard on the soundtracks of countless low-budget sci-fi movies. Most people 
have never heard of the Thereminvox, but nearly everyone has heard one played. In spite of its 'scary 
monster' associations, the Thereminvox (which is currently enjoying something of a comeback) is 
intended as a serious musical instrument. Repertoire exists for it, and the performances of Theremin 
virtuoso Clara Rockwell attest to the instrument's expressive power. 

Soundbeam's development was inspired by the Thereminvox. There are, however, two essential 
differences between the two machines. Firstly, whereas the Theremin creates a fixed playing zone close 
to the device itself, Soundbeam (designed for dancers) incorporates a variable ranging control which 
allows the invisible beam to be compressed into a few centimetres or stretched out to cover em entire 
stage area. In practice this means that the invisible instrument can be varied in size to accommodate the 
movements that the player wishes to perform, or is capable of performing. The second key difference is 
that whereas with the Theremin variations of timbre were not available (you were more or less stuck 
with the scary monster sound), Soundbeam - on its own - produces no sound at all. Instead, the machine 
functions as an information processor which translates distance and movement data into signals which 
are understood by electronic instruments, so that any sound which a given instrument will generate can 
be 'played' with the beam. 

Soundbeam works by emitting an invisible beam of high frequency sound (ultrasound) inaudible to 
human ears. The ultrasonic pulses are reflected back into the device's sensor by interruptions of and 
movements within the beam. Information about the speed and direction of this movement is translated 
into a digital code (MIDI) which is understood by a growing proliferation of electronic musical 
instruments, from relatively simple and inexpensive home keyboards to professional studio-quality 
synthesisers, samplers and expanders. 

The system incorporates three principal control parameters. Variations in Range settings allow the length 
of the beam to be varied between 0.25 and 6 metres. Shorter beams concentrate note information into a 
relatively small space, a set-up which has proved to be of significant value in special education where 
player's ability to move expansively may be limited by disability. Longer beams allow a complete 
performance space to be 'live' with sounds. Transpose settings allow semitonal modulation of the scales 
contained within the beam, useful where the beam is played alongside other electronic or acoustic 
instruments. Mode settings govern the realisation of note information contained in the beam. Each of the 
Mode presets comprises four variables: the number of notes potentiated (between one and sixty-four), 
the relationship of those notes to one another (scales, chords and arpeggios), the articulation required to 
activate the note (the dynamics of the movement in relation to the sensor), and secondary information 
such as velocity, pitchbend and modulation depth. In addition to the resident Soundbeam 'memories' it is 
also possible to load a user-defined sequence of notes into the beam. This has the benefit of enabling a 
considerable degree of compositional exploration without the necessity for a commensurate level of 
keyboard skill: an idea can be programmed in at the user's pace and then performed in real time with 
dance or simple movement. Up to four sensors can be attached to the basic system. Thus it is possible to 
have a three-dimensional playing space, with beams each of different length, each angled in a different 
plane, and each with its own sound or 'voice'. 
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SOUND THERAPY 



In traditional music therapy, the less the child is able to say something with sound because of a physical 
or cognitive disability, the heavier becomes the therapist's responsibility for empathy and interpretation . 
The main focus and engine for the mood and meaning of the music which is happening is on the 
therapist, and this creative and interpretative role is increasingly shifted away from the child with more 
profound levels of disability. Consequently, as the liberating potential of musical expression increases, it 
becomes correspondingly less achievable. This allocation of creative 'power' may have no clinical or 
therapeutic rationale, it may simply result from what is physically possible. New evidence suggests that 
Soundbeam can provide answers to this problem. 

The experience of initiation is central to the success of Soundbeam, especially for individuals with 
profound disabilities. If ones overall experience of life is essentially passive, it may be difficult to 
develop any concept of 'selfhood', any idea of oneself as a separate individual. What Soundbeam offers, 
perhaps for the first time and regardless of the individual's degree of immobility, is the power to make 
something happen. This is the vital experience of "that was me!", which can function as the foundation 
stone for further learning and interaction. This use of sound as the source of motivation is an extremely 
simple but crucially important application of the technology; it is impossible to overstate its value. 

Work by Dr.Phil Ellis (Ellis ‘94, ‘95, ‘96) at the Institute of Education at the University of Warwick in 
England has provided us with the first systematic long-term evaluation of Soundbeam's potential for 
children and adults with disabilities. The beam is positioned so that as soon as the child begins to move 
an interesting sound is triggered, motivating further movement and, eventually, radically enhanced 
posture, balance and trunk control. All of this is accomplished in parallel with a strong sense of fun and 
achievement. For the child, the therapeutic dimension of what is happening is irrelevant. Ellis also 
discusses some of the broader aesthetic issues connected with his approach: 

“There are differences between sound and music, but the term ‘music’ may now encompass a broader 
sound spectrum due to the possibilities which have emerged during this century through the increasing 
use of electricity in music. As early as 1932 Stokowski predicted a time when musicians would be able 
to compose directly into TONE, not on paper. In other words we would be working directly with sound 
itself rather than with the symbols used to represent the results of imagined combinations of sound. 
Furthermore, through sound synthesis electricity has made it possible to discover and create sounds 
which have never before been heard, and which could not be created any other way. In addition, we can 
simulate... a range of acoustic environments - concert halls, rooms, cathedrals or other large spaces for 
example, or may create acoustic environments which are impossible to encounter in the external physical 
world. This aural richness and variety provides the internal motivation which lies at the heart of ^s 
approach. In addition the technology also provides physical access for the disabled. 

....Sound itself is the medium of interchange... This approach contrasts with traditional models of music 
therapy, with its emphasis on ‘treatment’, direct intervention and imposition of external stimuli 
determined by an outside agent. Even where a music therapist may claim to be ‘responding’ to a 
patient’s music, this is a personal response on the part of the therapist. Often the therapist uses, or moves 
towards, a traditionally based musical language comprised of melody, harmony and rh 5 dhm, so limiting 
the soundscape and genre of ‘musical’ discourse. The ‘patient’ or ‘client’ is viewed in a clinical way, 
with a condition which needs to be treated or ameliorated. There are clearly defined goals with these 
treatments, with success measured according to how effective the treatment has been in terms of the 
clinical or medical condition. The modus operandi of these approaches is essentially from the outside 
-in, with an emphasis on clinical intervention rather than independent learning. 

In Sound Therapy a different, contrasting approach is taken. Whilst progression and development remain 
a key focus, the essence lies in the internal motivation of the child, in working fi-om the inside - out. This 
internal motivation is produced through the use of sound within a carefully controlled environment. At 
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all times the child is given the opportunity to independently take control of the situation as far as 
possible. Certain aspects are controlled externally - notably the sonic environment - but the essence lies 
in allowing the child freedom to act as she or he chooses within available parameters which remain as 
open as possible. In this way, learning occurs incidentally. As a result we can see progression and 
development in a variety of ways across a range of disabilities. Such progression is not prescribed in 
advance, but happens as a natural and additional part of activity, all stenuning from the internal 
motivation of the child - a phenomenon referred to as aesthetic resonation. This is made possible 
through a particular use of sound as the primary medium of interaction, and through giving access 
through the use of technology, so enabling even profoundly handicapped children the opportunity for 
expression and control - in other words the encounter with and development of communication skills - 
through sound.” 

From systematic analysis of videotape session records. Dr. Ellis has identified nine criteria of 
progression and development: 

1 . from involuntary to voluntary 

2. from accidental to intended 

3. from indifference to interest 

4. from confined to expressive 

5. from random to purposeful 

6. from gross to fine 

7. from exploratory to preconceived 

8. from isolated to integrated 

9. from solitary to individual 

He notes that even profoundly disabled children respond to Sound Therapy by: 

• performing, listening, verbalising, 'composing' with sound; 

• often showing 'aesthetic resonance' through most telling facial expressions; 

• being actively involved for extended periods of time; 

• revealing an ability for concentration not apparent elsewhere; 

• beginning to discover, explore, give expression to and communicate their own feelings; 

• m^ing significant physical responses - movements and gestures which hitherto have not been 
seen, or have not previously been made independently. 

"...in addition, a change has been seen in behaviour patterns beyond the inunediate 
environment of Sound Therapy. Some children are now more self-aware and are 
interacting.. .Other children show more tolerance and a growing awareness of other people, 
moving towards interpersonal skills." 

In these early explorations of the beam the therapist has two important jobs. Firstly, to select from the 
(probably hundreds) of available sounds, interesting and motivating ones which will stimulate interest 
(this is particularly important where cheaper synthesisers are used as these will contain many 
uninteresting and unpleasant sounds), and to sustain engagement by changing synthesiser voices and, 
possibly, Soundbeam modes so that musical output does not become tedious; and secondly, to observe 
and estimate the various axes and degrees of physical movement so that the 'invisible sounds' are 
optimally positioned in relation to the child's body. 

“.... few of these children have the necessary physical co-ordination or control necessary for 
traditional musical performance. However, we can legitimately shift the view away from 
purely traditional musical qualities towards the new and developing musical aesthetic, one 
which has in part been enabled through the introduction of electricity to musical activity. In 
so doing many musical doors can be opened and we can all enjoy being expressive with 
sound... Many of these techniques can be made easily accessible to virtually all children 
through technology.” 

Kathryn Russell (Russell ‘96), working in Australia has also identified a number of areas of 



development with the special students she involves with Soundbeaim, including: 

• AESTHETIC AWARENESS (includes the capacity to make choices and judgements as to what 
sounds or movements to select and manipulate) 

• IMAGINATION (anticipating sounds and movements well ahead of time - perhaps in the week 
between classes) 

• LISTENING SIGLLS (listening to the effects of moving or standing still) 

• CHOICE-MAKING SKILLS (will I choose Soundbeaun? which sound will I choose? which part 
of my body will I move? where will I move?) 

• CONCEPTUAL SKILLS (especially ‘beginning, middle and end’ - how will I begin? what will I 
do then? how will I make an ending? specific musical concepts such as high, low, fast, slow, 
variation; the concepts of linkage - words with movement, feeling with tone colour...) 

• MOTOR PLANNING SKILLS (which movement will I make now to produce...?) 

• REFLECTIVE COGNITION (how did I feel about the piece I just invented? what could I have 
done differently? What did it remind me of?) 

• MEMORY SKILLS (can I remember which sound I liked last time? do I want to use it again?) 

• SPATIAL ORIENTATION (where in space is that dog barking sound?) 

• LANGUAGE SKILLS (describing what I did and how I felt, giving a title to my work). 

• EXPLORING A HYPOTHESIS (I remember that if I move this way, that sound happens. If I 
move the same way, will I get the same sound?) 

• SOCIAL SKILLS (waiting for a solo turn, sharing the beam to produce joint improvisations). 

• CONFIDENCE (this is something that I can do). 

“Bearing in mind the extremely short attention span of many children with special needs, 
students have demonstrated a remarkable capacity to focus on their improvisations for long 

periods of time, thought previously to be beyond their abilities Those using Soundbeaun 

for music education have discovered that children who are able to take control of their 
music making develop not only expressive and practical movement capabilities, but also 
create improvisatory music which has relevance and validity” 



FROM THERAPY TO PERFORMANCE 

Russel’s findings, like those of Dr. Ellis, provide a systematic assessment of physical and affective 
responses, giving quantifiable data about Soundbeaun's clinical possibilities. In parallel with this 
approach, many projects in the disability field are exploring the creative performance-based paths 
opened up by this new technology. At 'The Ark' in Bracknell, a multi-arts project for people with 
learning difficulties, dancer Penny Sanderson and musician David lackson's workshops always involve a 
narrative theme, and include live music, dance and drama, providing an excellent exaunple of the way in 
which technology can be used to complement and enhance a successful established activity (rather than a 
sterile 'technology-led' approach) in a way which allows participants a fuller involvement. The sounds 
and tunes triggered by the students' movements in the beaun have a place in the story and are tailored to 
the personalities and moods of the individual students. Dinosaurs, waterfalls, butterflies, monsters, 
princesses, lions, explosions - this aural dimension is all controlled and modulated by the students’, and 
this in turn reinforces and remotivates their involvement. 

Fransisco Borges da Souza, a music therapist working in Portugal, has formed a rock band with 
Soundbeam as one of the key instruments. The astounding expressiveness of the disabled Soundbeam 
player clearly reveals a talent that has been unlocked. 

Special schools in England are now starting to collaborate with so-called ‘mainstream’ schools on music 
projects involving Soundbeaun. As the children are able to learn and perform on an equal basis, the 
disabled/non-disabled barriers can be broken down. English special schools are often isolated and the 
arts can clearly give a strong focus for integration where this can be enabled by appropriate technology. 
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Meldreth Manor School in Hertfordshire and the Ormerod School in Oxford are outstanding examples. 



THE SOUNDBEAM PROJECT 

We are taught that 'serious' musicianship demands years of dedication. So what are we to make of 
devices which allow musical expression to happen almost immediately? How can we asses the musical 
validity of what we hear? With conventional instruments, designed for those with average or 
above-average physical, mental and sensory functioning, the time gap between musical imagination and 
musical realisation takes years to develop. Good technology radicdly shortens this gap. It extends the 
limits of selected-scale or percussion based work, and it asks the player to learn not the technical skills 
of the traditional instrumentalist but the freedoms and disciplines of improvisation. This kind of music is 
difficult to evaluate because there are no right or wrong ways of playing it - no performance of a piece of 
music played with Soundbeam will ever sound the same twice; but it is possible to assess the extent to 
which the student enjoys it and gets a feeling of achievement from it, and some of the preliminary 
research reviewed here indicates strongly that the attainment of significant milestones in the physical, 
cognitive and social development of individuals with a range of disabilities can be radically assisted by 
the use of such technology. 
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